Software developer
The main concern in AWS Glue is to so much costing of Glue jobs and I was worked with 5000 dataset I was facing some performance issue as …
Starting at $0.44 billed per second, 1 minute minimum
View PricingOverview
What is AWS Glue?
AWS Glue is a managed extract, transform, and load (ETL) service designed to make it easy for customers to prepare and load data for analytics. With it, users can create and run an ETL job in the AWS Management Console.…
Recent Reviews
Reviewer Pros & Cons
Engineer in Information Technology
Curitics health (Health, Wellness and Fitness, 201-500 employees)
Pricing
per DPU-Hour
$0.44
Cloud
billed per second, 1 minute minimum
Entry-level set up fee?
- No setup fee
Offerings
- Free Trial
- Free/Freemium Version
- Premium Consulting/Integration Services
Product Details
- About
- Tech Details
What is AWS Glue?
AWS Glue Technical Details
| Deployment Types | Software as a Service (SaaS), Cloud, or Web-Based |
|---|---|
| Operating Systems | Unspecified |
| Mobile Application | No |
Comparisons
Compare with
Reviews and Ratings
(31)Attribute Ratings
Reviews
(1-4 of 4)Companies can't remove reviews or game the system. Here's why
July 25, 2023
AWS Glue ETL tool
We use AWS Glue to creat Etl pipelines for transforming and moving of data from different data sources like S3, snowflakes, postgres to Redshift and vice versa. Execution of spark jobs is really easy as it has auto generated code which establishes connections with source and target data bases securedly and helps in the cleansing of data like deduplication and performing validations on data. As it is Serverless it will automatically scale up and scale down the memory resources required to run the spark glue job.
- Execution of spark jobs
- Scaling of memory resources
- Crawling the schemas
- Incremental data sync
- Real time data triggers
- Grouping of small files
- Extract
- Transform
- Load
- Spark jobs
- Crawling
- Connections
- Easy to use
- Lesser training required
- Detailed documentation available hence implementation is easy
AWS Glue is easier to use and has more and better features compared to it. And more documentation and tutorials and labs are widely available on the internet about AWS Glue which in turn helps in easier implementation of the spark jobs. Auto scaling is an added advantage. It's Serverless features will make it stand out from others.
September 20, 2022
Great for ETL and batch processing
1) In my current use case we mainly use AWS Glue for Extract Transform Load to process batch data on daily basis. 2) the main problem we can able to solve or we can say the solution which Glue provides that is, it can easily integrate with other AWS services like S3, RDS, Athena 3) pricing model is also very like pay-as-you-go 4) the main business problem which glue solve we can ingest the data ewe can perform ETL on top that and can create spark or python shell jobs
- Extract , transform , Load
- AWS Data catalog
- triggers
- we can create workflows
- In-Stream schema registries feature people can not use this more efficiently
- in Connections feature they can add more connectors as well
- The crucial problem with AWS Glue is that it only works with AWS.
- AWS Glue studio
- crawler
- notebook
- triggers
- workflows
- Positive Impact :- after ETL we can able to do some kind of automation
- Negative :- At some point of time it can hamper the cost but not really
The main reason we choose AWS Glue over Talend Open Studio 1) Does not support Spark 2) Run only on java 3) not really feasible solution for heavy workloads 4) most of the cases need customer support 5) no proper documentation is available
July 03, 2022
AWS Glue - The managed ETL service for your data
We use AWS Glue for ETL of the healthcare data. The input data come from different source systems and so with different formats. With help of the AWS Glue jobs, we translate the data into a common format. With help of python scripts and the scheduled job feature, the data is fetched in a periodic manner, processed with help of the python script, converted to the parquet format, and stored in the S3 bucket. The glue catalog generates the schema of the stored data and allows AWS Athena to query the same for analytics purposes.
- It is extremely fast, easy, and self-intuitive. Though it is a suite of services, it requires pretty less time to get control over it.
- As it is a managed service, one need not take care of a lot of underlying details. The identification of data schema, code generation, customization, and orchestration of the different job components allows the developers to focus on the core business problem without worrying about infrastructure issues.
- It is a pay-as-you-go service. So, there is no need to provide any capacity in advance. So, it makes scheduling much easier.
- The sample code should cover more scenarios. They are quite basic. However, you can find good pointers from the internet and AWS community and tickets.
- AWS Glue runs on Apache Spark. So, to take the best of the AWS Glue service, the developer should have a good idea of Apache Spark.
- AWS Glue Data catalog to write the efficient queries.
- AWS Glue Crawler for the automatic schema recognition.
- AWS Glue schedule job to perform certain ETL tasks on the defined interval.
- We were transforming the data using a simple python script and were facing a lot of orchestration issues. The failure of the script was quite prominent as the nature of the data was a bit more dynamic. With help of AWS glue, we could fix ~80% of orchestration issues. With help of automatic schema generation, dynamism is also addressed very well. So, we have started realising the ROI from day 1.
Glue comes in form of a managed service. However, the AWS Data Pipeline puts additional responsibility to manage the infrastructure. We were not requiring fine-grained control of the hardware which the AWS Data Pipeline provides. We also want to park our data on DynamoDB. AWS Glue allows storing the data to DynamoDB but the same is not possible with the AWS Data Pipeline. So, we decided to move ahead with AWS Glue.
January 13, 2020
AWS Glue : a fully managed ETL service
One of the straightforward and quick cloud-based ETL tools is AWS Glue. It comes under the umbrella of AWS services. We use AWS Glue to analyze an extensive data set of USA based clinics and hospitals. Its HIPAA compliance for sensitive data. It comes with the support of python script, Schedular, and works very well with other AWS services like s3, rds.
- Very quick for ETL job.
- UI as well Command Interface with very few steps to create and schedule ETL Job.
- Sample Code is very basic and not available in most of the scenario.
- Helps to leverage the data in realtime.
- Leads to quick and clear business decision making.
We are already in AWS services, so AWS glue is the first choice for us. But for the comparison of ETL job making and process time, it's way faster for other services.